The development of deep learning models in medical image analysis is majorly limited by the lack of large-sized and well-annotated datasets. Unsupervised learning does not require labels and is more suitable for solving medical image analysis problems. However, most of the current unsupervised learning methods need to be applied to large datasets. To make unsupervised learning applicable to small datasets, we proposed Swin MAE, which is a masked autoencoder with Swin Transformer as its backbone. Even on a dataset of only a few thousand medical images and without using any pre-trained models, Swin MAE is still able to learn useful semantic features purely from images. It can equal or even slightly outperform the supervised model obtained by Swin Transformer trained on ImageNet in terms of the transfer learning results of downstream tasks. The code will be publicly available soon.

Gaussian process state-space model (GPSSM) is a fully probabilistic state-space model that has attracted much attention over the past decade. However, the outputs of the transition function in the existing GPSSMs are assumed to be independent, meaning that the GPSSMs cannot exploit the inductive biases between different outputs and lose certain model capacities. To address this issue, this paper proposes an output-dependent and more realistic GPSSM by utilizing the well-known, simple yet practical linear model of coregionalization (LMC) framework to represent the output dependency. To jointly learn the output-dependent GPSSM and infer the latent states, we propose a variational sparse GP-based learning method that only gently increases the computational complexity. Experiments on both synthetic and real datasets demonstrate the superiority of the output-dependent GPSSM in terms of learning and inference performance.

Vision Transformers (ViTs) outperforms convolutional neural networks (CNNs) in several vision tasks with its global modeling capabilities. However, ViT lacks the inductive bias inherent to convolution making it require a large amount of data for training. This results in ViT not performing as well as CNNs on small datasets like medicine and science. We experimentally found that masked autoencoders (MAE) can make the transformer focus more on the image itself, thus alleviating the data-hungry issue of ViT to some extent. Yet the current MAE model is too complex resulting in over-fitting problems on small datasets. This leads to a gap between MAEs trained on small datasets and advanced CNNs models still. Therefore, we investigated how to reduce the decoder complexity in MAE and found a more suitable architectural configuration for it with small datasets. Besides, we additionally designed a location prediction task and a contrastive learning task to introduce localization and invariance characteristics for MAE. Our contrastive learning task not only enables the model to learn high-level visual information but also allows the training of MAE's class token. This is something that most MAE improvement efforts do not consider. Extensive experiments have shown that our method shows state-of-the-art performance on standard small datasets as well as medical datasets with few samples compared to the current popular masked image modeling (MIM) and vision transformers for small datasets.The code and models are available at https://github.com/Talented-Q/SDMAE.

Traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples. This paradigm seriously hinders the development of task generalization since preparing a task-specific example set is costly. To build a system that can quickly and easily generalize to new tasks, task instructions have been adopted as an emerging trend of supervision recently. These instructions give the model the definition of the task and allow the model to output the appropriate answer based on the instructions and inputs. However, task instructions are often expressed in different forms, which can be interpreted from two threads: first, some instructions are short sentences and are pretrained language model (PLM) oriented, such as prompts, while other instructions are paragraphs and are human-oriented, such as those in Amazon MTurk; second, different end-users very likely explain the same task with instructions of different textual expressions. A robust system for task generalization should be able to handle any new tasks regardless of the variability of instructions. However, the system robustness in dealing with instruction-driven task generalization is still unexplored. This work investigates the system robustness when the instructions of new tasks are (i) maliciously manipulated, (ii) paraphrased, or (iii) from different levels of conciseness. To our knowledge, this is the first work that systematically studies how robust a PLM is when it is supervised by instructions with different factors of variability.

Facial expression recognition (FER) plays a significant role in the ubiquitous application of computer vision. We revisit this problem with a new perspective on whether it can acquire useful representations that improve FER performance in the image generation process, and propose a novel generative method based on the image inversion mechanism for the FER task, termed Inversion FER (IFER). Particularly, we devise a novel Adversarial Style Inversion Transformer (ASIT) towards IFER to comprehensively extract features of generated facial images. In addition, ASIT is equipped with an image inversion discriminator that measures the cosine similarity of semantic features between source and generated images, constrained by a distribution alignment loss. Finally, we introduce a feature modulation module to fuse the structural code and latent codes from ASIT for the subsequent FER work. We extensively evaluate ASIT on facial datasets such as FFHQ and CelebA-HQ, showing that our approach achieves state-of-the-art facial inversion performance. IFER also achieves competitive results in facial expression recognition datasets such as RAF-DB, SFEW and AffectNet. The code and models are available at https://github.com/Talented-Q/IFER-master.

We present DualNER, a simple and effective framework to make full use of both annotated source language corpus and unlabeled target language text for zero-shot cross-lingual named entity recognition (NER). In particular, we combine two complementary learning paradigms of NER, i.e., sequence labeling and span prediction, into a unified multi-task framework. After obtaining a sufficient NER model trained on the source data, we further train it on the target data in a {\it dual-teaching} manner, in which the pseudo-labels for one task are constructed from the prediction of the other task. Moreover, based on the span prediction, an entity-aware regularization is proposed to enhance the intrinsic cross-lingual alignment between the same entities in different languages. Experiments and analysis demonstrate the effectiveness of our DualNER. Code is available at https://github.com/lemon0830/dualNER.

最近，基于云的图形卷积网络（GCN）在许多对隐私敏感的应用程序（例如个人医疗保健和金融系统）中表现出了巨大的成功和潜力。尽管在云上具有很高的推理准确性和性能，但在GCN推理中保持数据隐私，这对于这些实际应用至关重要，但仍未得到探索。在本文中，我们对此进行了初步尝试，并开发了$ \ textit {cryptogcn} $ - 基于GCN推理框架的同型加密（HE）。我们方法成功的关键是减少HE操作的巨大计算开销，这可能比明文空间中的同行高的数量级。为此，我们开发了一种方法，可以有效利用GCN推断中基质操作的稀疏性，从而大大减少计算开销。具体而言，我们提出了一种新型的AMA数据格式方法和相关的空间卷积方法，该方法可以利用复杂的图结构并在HE计算中执行有效的矩阵矩阵乘法，从而大大减少HE操作。我们还开发了一个合作式框架，该框架可以通过明智的修剪和GCN中激活模块的多项式近似来探索准确性，安全级别和计算开销之间的交易折扣。基于NTU-Xview骨架关节数据集，即，据我们所知，最大的数据集对同型的评估，我们的实验结果表明，$ \ textit {cryptogcn} $均优胜于最先进的解决方案。同构操作的延迟和数量，即在延迟上达到3.10 $ \ times $加速，并将总代态操作数量减少77.4 \％，而准确度的较小精度损失为1-1.5 $ \％$。

我们考虑了自动生成音乐文本描述的新颖任务。与其他完善的文本生成任务（例如图像标题）相比，富裕的音乐和文本数据集的稀缺性使其成为更具挑战性的任务。在本文中，我们利用众包音乐评论来构建一个新的数据集，并提出一个序列到序列模型以生成音乐的文本描述。更具体地说，我们将扩张的卷积层用作编码器的基本组成部分，基于内存的复发性神经网络作为解码器。为了增强生成文本的真实性和主题，我们进一步建议用歧视者和新的主题评估者微调模型。为了衡量生成的文本的质量，我们还提出了两个新的评估指标，它们比人类评估比传统指标（例如BLEU）更加一致。实验结果验证了我们的模型能够在包含原始音乐的主题和内容信息的同时产生流利而有意义的评论。

与自然图像相比，医学图像很难获取，标签成本很高。作为一种无监督的学习方法，对比学习可以更有效地利用未标记的医学图像。在本文中，我们使用了一种基于变压器的对比学习方法，并通过转移学习创新了对比度学习网络。然后，将输出模型转移到下游腮腺分割任务，该任务改善了测试集上腮腺分割模型的性能。改善的DSC为89.60％，MPA为99.36％，MIOU为85.11％，HD为2.98。与使用监督学习模型作为腮腺分割网络的预训练模型的结果相比，所有四个指标均显示出显着改善。此外，我们发现，通过对比度学习模型对细分网络的改进主要在编码器部分中，因此本文还试图为解码器部分构建对比度学习网络，并讨论了在构建过程中遇到的问题。

无监督的视频域适应是一项实用但具有挑战性的任务。在这项工作中，我们第一次从脱离视图中解决了它。我们的关键想法是在适应过程中将与域相关的信息从数据中删除。具体而言，我们考虑从两组潜在因素中生成跨域视频，一个编码静态域相关信息，另一个编码时间和语义相关的信息。然后开发转移顺序的VAE（Transvae）框架以建模这种产生。为了更好地适应适应，我们进一步提出了几个目标，以限制Transvae中的潜在因素。与几种最先进的方法相比，对UCF-HMDB，小丑和Epic-Kitchens数据集进行了广泛的实验验证了Transvae的有效性和优势。代码可在https://github.com/ldkong1205/transvae上公开获取。